At the age of 18 he fell in love with a farmer's daughter who was three years his senior and they were married. His wife, Margaret, was the daughter of Timothy Whalan and Margaret Kelley. They had eight children, three of whom died as infants. The survivors were: Mary (c1807 - 1880), Ellen (c1812 - 1894), James (c1815 - 1887), Judith (c1816 - 1883) and Patrick (c1823 - 1892).
It is well known that the lives of Irish farm workers were incredibly difficult. Patrick was arrested in early 1822, aged 34, and tried at the Spring assizes in Waterford. He was found guilty and transported to Australia on a life sentence. When he embarked on the Brampton on 8 November 1822, he was described as 5' 3½", of sallow complexion with dark brown hair and grey eyes. He was one of 172 convicts that disembarked in Sydney five months later, along with a detachment of the third regiment.
Somehow Margaret Grady struggled on in Tipperary without her husband, but he wanted her and the children to be with him. Just over a year after his arrival in Australia, he arranged for Thomas and Samuel Hassall to write to the Colonial Secretary with a petition that his family be allowed to join him in Australia. Patrick had been assigned to Reverend Thomas Hassell as a convict servant and resided at Bathurst. The letter read:
Sir
Patrick Grady, who arrived by the Brampton Captn Moore in 1823 Sentence Life is anxious to have his wife & five children sent out to him & begs that you will have his case taken into consideration and if pofsible afford him the pleasure of once more beholding & comforting all that is now near & dear to him.
We believe him to be a steady honest and industrious character & think he would be able to support the said wife & family if here without any expence to Government. I have the honour to be
Your most obedient
humble servant
Thos & Saml Hafsall
The letter indicated that the family lived at New Castle Parish Clon(mell), County Tipperary, Ireland and listed his children as James, Patrick, Mary, Elenor and Jude.
This request was granted and the family embarked on the City of Edinburgh on 23 June 1828 and arrived at Sydney five months later. By November of that year, Patrick had been joined at Bathurst by his son James but his wife and remaining children remained in Sydney. The following year Ellen married Patrick Foran, a 31 year old convict serving a life sentence. Mary married 24 year old Samuel Rawson, also a convict serving a seven year sentence. Judith married Thomas Casey (later Kessey) in 1832 and James married Rose Donnelly in 1835.
Patrick was awarded a Ticket-of-leave in early 1832 and soon afterwards the whole clan moved to the Brisbane Valley area on the Fish River near Oberon in NSW. The extended clan (the Forans, Kesseys and Donnellys) had many brushes with the law over the next few decades.
Patrick's wife, Margaret died and was buried on 7 June 1857, aged 72. A burial witness was Charles Whalan, but it is not clear that he was a relative. He could possibly be a son of the Charles Whalan who was sentenced to transportation in 1787 (when Margaret was just 2 years old) and arrived in 1791 aboard Albermarle. He enlisted in the NSW corps and became Sergeant of the Guard during Governor Macquaries' time. Charles Whalan junior too is buried in the area.
Three years later, at the age of 72, Patrick married Susan Ryan, a widow whose maiden name was Harty. There were no children. Patrick died six years later at O'Connell, near Bathurst, aged 80.